[refine](exec/operator) replace std::mutex/std::lock_guard with annotated wrappers for thread safety analysis#63070
Merged
BiteTheDDDDt merged 2 commits intoMay 9, 2026
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Mryange
force-pushed
the
use-AnnotatedMutex-for-pipeline
branch
from
7f97364 to
a29fb60
Compare
Mryange
changed the title
Mryange
changed the title
Contributor
Author
|
/review |
Contributor
|
OpenCode automated review failed and did not complete. Error: Review step was failure (possibly timeout or cancelled) Please inspect the workflow logs and rerun the review after the underlying issue is resolved. |
Contributor
Author
|
/review |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Review opinion: no blocking issues found in the changed PR diff.
Critical checkpoint conclusions:
- Goal/test: The PR appears focused on adding annotated mutex/lock wrappers and applying them to selected pipeline shared-state locks so Clang thread-safety annotations can be used. The existing multicast streamer test was adjusted to respect the new guarded fields; I did not run tests in this review environment.
- Scope/focus: The change is small and focused on lock annotation/conversion in the authoritative PR patch.
- Concurrency: Reviewed the converted pipeline locks, dependency interactions, multicast spill/read paths, shared hash table finish dependency list, exchange finished-channel set, and scan conjunct lock. I did not find a concrete introduced race, missed wake-up, or deadlock from the changed lock wrappers/usages.
- Lifecycle/static initialization: No new static/global lifecycle dependency was introduced.
- Configuration/compatibility: No new config items, wire protocol changes, persisted format changes, or rolling-upgrade compatibility concerns were found.
- Parallel paths: The modified paths are localized conversions; no obvious parallel path requires the same annotation change for correctness.
- Conditional checks/error handling: No new unchecked Status or silent error continuation was found in the changed code.
- Test coverage/results: Existing test coverage is lightly adjusted for guarded access; no new behavioral test was required by the annotation-only intent. I did not verify by running BE tests.
- Observability: No new runtime behavior requiring additional logs or metrics was introduced.
- Transactions/data writes/FE-BE passing: Not applicable to this PR diff.
- Performance: The annotated wrappers preserve std::mutex semantics at reviewed call sites; no obvious hot-path extra work is introduced outside BE_TEST mock sleeps.
User focus: no additional user-provided review focus was present.
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29606 ms |
Contributor
TPC-DS: Total hot run time: 169778 ms |
Contributor
Author
|
run p0 |
Contributor
Author
|
run cloud_p0 |
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
BiteTheDDDDt
approved these changes
May 9, 2026
Mryange
added a commit
to Mryange/doris
that referenced
this pull request
May 15, 2026
…ated wrappers for thread safety analysis (apache#63070) This PR introduces Clang thread safety annotations (-Wthread-safety) to pipeline operator shared states by replacing raw std::mutex/std::lock_guard/std::unique_lock with annotated wrappers (AnnotatedMutex, LockGuard, UniqueLock), and by decorating guarded member variables with GUARDED_BY attributes. This enables the compiler to statically detect data races where a field is accessed without holding its associated mutex. The change also fixes two bugs uncovered during the annotation process: - MultiCastDataStreamer::push: _eos (member) was checked instead of eos (parameter), causing the "set always ready" branch to fire on the prior call's stale state rather than the current one. - MultiCastDataStreamer::pull's spill lambda: _cached_blocks[sender_idx].empty() was checked outside the mutex; the check is now done via a boolean captured inside the lock. (cherry picked from commit 2dc0d0a)
16 tasks
zclllyybb
pushed a commit
that referenced
this pull request
May 18, 2026
…ted wrappers for thread safety analysis (#63109) After #63070 replaced `std::mutex`/`std::lock_guard` with annotated wrappers (`AnnotatedMutex`/`LockGuard`), shared mutex usage still relied on raw `std::shared_mutex` and `std::shared_lock`/`std::unique_lock`, which are invisible to Clang's thread safety analysis. This leaves shared-lock sites unverified — `GUARDED_BY`, `REQUIRES_SHARED`, and other annotations cannot be enforced. - Add `AnnotatedSharedMutex` (wrapping `std::shared_mutex` with `CAPABILITY`/`ACQUIRE`/`RELEASE`/`ACQUIRE_SHARED`/`RELEASE_SHARED` annotations) and `SharedLockGuard` (RAII `SCOPED_CAPABILITY` with `ACQUIRE_SHARED`/`RELEASE`) in `thread_safety_annotations.h`. - Migrate `VDataStreamMgr` and `RuntimeFilterMergeControllerEntity` from `std::shared_mutex` to `AnnotatedSharedMutex`, and from `std::unique_lock`/`std::shared_lock` to `LockGuard`/`SharedLockGuard`. - Add `GUARDED_BY` annotations to the protected maps. - Extract `_find_recvr` as a private helper annotated with `REQUIRES_SHARED(_lock)`, eliminating the `bool acquire_lock` parameter that previously bypassed lock tracking.
yiguolei
pushed a commit
that referenced
this pull request
May 20, 2026
### What problem does this PR solve? #62947 #63055 #63070 ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
What problem does this PR solve?
Issue Number: N/A
Problem Summary:
This PR introduces Clang thread safety annotations (-Wthread-safety) to pipeline operator shared states by
replacing raw std::mutex/std::lock_guard/std::unique_lock with annotated wrappers (AnnotatedMutex, LockGuard,
UniqueLock), and by decorating guarded member variables with GUARDED_BY attributes. This enables the compiler to
statically detect data races where a field is accessed without holding its associated mutex.
The change also fixes two bugs uncovered during the annotation process:
branch to fire on the prior call's stale state rather than the current one.
check is now done via a boolean captured inside the lock.
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)